feat(workflows): design workflow to run dataverse in a container#87
feat(workflows): design workflow to run dataverse in a container#87srmanda-cs wants to merge 7 commits intouncch-rdmc:developfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a new GitHub Actions workflow intended to spin up the Dataverse dev Docker Compose stack in CI on pushes/PRs targeting develop, acting as a lightweight “container comes up” check.
Changes:
- Introduces
.github/workflows/container_testing_run.ymlto build/start thedocker-compose-dev.ymlstack onubuntu-latest. - Adds a basic runtime check via
docker psand always-runs teardown/cleanup. - Includes steps to scrub and reinstall Docker Engine on the runner before running Compose.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| sudo docker ps | ||
| RUNNING=$(sudo docker ps -q | wc -l) | ||
| if [ "$RUNNING" -eq 0 ]; then | ||
| echo "No containers are running." | ||
| exit 1 | ||
| fi | ||
| echo "Containers running: $RUNNING" | ||
|
|
There was a problem hiding this comment.
The container verification only checks that some containers are running. This can pass even if the main Dataverse container exits but dependencies (e.g., Postgres/Solr) are still up. Consider asserting that the expected service(s) (at least dev_dataverse) are running/healthy (e.g., docker compose ps --status running for a specific service, or inspect the container state/health).
| sudo docker ps | |
| RUNNING=$(sudo docker ps -q | wc -l) | |
| if [ "$RUNNING" -eq 0 ]; then | |
| echo "No containers are running." | |
| exit 1 | |
| fi | |
| echo "Containers running: $RUNNING" | |
| echo "Listing all Docker Compose services:" | |
| sudo docker compose -f docker-compose-dev.yml ps | |
| echo "Checking that dev_dataverse service is running..." | |
| RUNNING_SERVICES=$(sudo docker compose -f docker-compose-dev.yml ps --status running --services) | |
| if ! echo "$RUNNING_SERVICES" | grep -q '^dev_dataverse$'; then | |
| echo "Expected service 'dev_dataverse' is not running." | |
| echo "Currently running services:" | |
| echo "$RUNNING_SERVICES" | |
| exit 1 | |
| fi | |
| echo "Verified: 'dev_dataverse' service is running." |
| - name: Scrub existing Docker installations | ||
| run: | | ||
| set -euo pipefail | ||
| sudo systemctl stop docker || true | ||
| sudo apt-get remove -y docker docker-engine docker.io containerd runc || true | ||
| sudo apt-get purge -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin || true | ||
| sudo rm -rf /var/lib/docker | ||
| sudo rm -rf /var/lib/containerd | ||
|
|
There was a problem hiding this comment.
The workflow fully removes and reinstalls Docker/Containerd on every run. This is a large, failure-prone setup step and diverges from the repo’s other container workflows (which rely on the runner’s preinstalled Docker and/or docker/* setup actions). Consider dropping the scrub/reinstall and just ensure Docker is available (or use docker/setup-buildx-action / docker/setup-qemu-action if needed).
| on: | ||
| push: | ||
| branches: | ||
| - develop | ||
| paths-ignore: | ||
| - 'doc/**' | ||
| - '**/*.md' | ||
| - '**/*.txt' | ||
| - '.github/ISSUE_TEMPLATE/**' | ||
| - '.github/*.md' | ||
| pull_request: | ||
| branches: | ||
| - develop | ||
| paths-ignore: | ||
| - 'doc/**' | ||
| - '**/*.md' | ||
| - '**/*.txt' | ||
| - '.github/ISSUE_TEMPLATE/**' | ||
| - '.github/*.md' |
There was a problem hiding this comment.
This workflow runs on every push/PR to develop except docs/markdown/text changes. Given it pulls/builds and starts a full dev compose stack, this will add significant CI load for unrelated changes. Consider adding a paths: filter to only run when container/compose-related files change (e.g., docker-compose-dev.yml, docker/**, src/main/docker/**, etc.).
| - name: Build containers | ||
| run: | | ||
| set -euo pipefail | ||
| sudo docker compose -f docker-compose-dev.yml build | ||
|
|
There was a problem hiding this comment.
docker compose ... build won’t build anything with the current docker-compose-dev.yml because that compose file contains no build: sections (only image:). This makes the step misleading and doesn’t validate the PR’s code. Either remove the build step, or update the compose workflow to actually build from the checked-out sources (and set APP_IMAGE accordingly).
| - name: Build containers | |
| run: | | |
| set -euo pipefail | |
| sudo docker compose -f docker-compose-dev.yml build |
| - name: Launch containers | ||
| run: | | ||
| set -euo pipefail | ||
| sudo docker compose -f docker-compose-dev.yml up -d | ||
|
|
There was a problem hiding this comment.
The compose file relies on ${STORAGE_DIR} (used in JVM_ARGS) but this workflow doesn’t set it and it’s not present in the repo’s committed .env. In CI this will substitute to an empty string (effectively /store), which can make container startup/config non-deterministic. Set STORAGE_DIR explicitly (e.g., via an env: block for the job/step) or change the compose file to provide a default (${STORAGE_DIR:-...}).
What this PR does / why we need it:
Which issue(s) this PR closes:
Special notes for your reviewer:
Suggestions on how to test this:
Does this PR introduce a user interface change? If mockups are available, please link/include them here:
Is there a release notes update needed for this change?:
Additional documentation: